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Executive Summary 

West Virginia Code §i8-2-23a 1 required the West Virginia Board of Education 
(WVBE) to establish annual professional development goals for public schools; to coordinate 
professional development programs; and to guide program development, approval and eval- 
uation. Toward these ends, the WVBE (2012) adopted the following goals for professional 
development for the 2012-2013 school year: 

To provide professional development that— 

1. Aligns with curriculum standards to increase educator effectiveness in the arts, 
world languages, health, physical education, career/technical, reading/ English 
language arts, mathematics, science, and social studies. 

2. Focuses on developing in-depth understanding of the essential features of the 
growth and development of the personal, physical, social, and emotional needs of 
each student, including providing students with personalized pathways and guid- 
ance to help them have productive and satisfying lives. 

3. Develops the leadership competencies, professional culture, and characteristics 
necessary to increase the support of teaching and learning. 

West Virginia Code §l8-2-23a further required that, each year, once the annual goals 
are set, the state board must submit the goals to the major state agencies responsible for 
providing professional development to teachers, administrators, and other professional edu- 
cation staff statewide, including the West Virginia Department of Education (WVDE), the 
West Virginia Center for Professional Development (CPD), the regional education service 
agencies (RESAs), and the Higher Education Policy Commission (HEPC). These agencies 
then are required to collaborate in the development of an annual master plan for profession- 
al development aligned with the goals. Additionally, the statute requires evaluation of the 
effectiveness of the professional staff development programs. The WVBE charged the WVDE 
Office of Assessment, Accountability, and Research to meet this requirement. 

In this evaluation, as in previous years, we examined four main aspects of the im- 
plementation of the West Virginia Board of Education’s Master Plan for Statewide Profes- 
sional Development: (a) basic information reported by providers about the size and scope of 
the effort, including attendance, and adherence to the newly adopted standards for profes- 
sional development; and participant reports (gathered through the administration of a ran- 
domized statewide survey) about the (b) quality of the sessions, (c) their alignment to Board 
goals for professional development, and (d) the impacts of the sessions on participants’ 
knowledge, practice, and attitudes and beliefs. Each of these four areas is discussed below, 
including trends noted over the three years that the WVDE Office of Assessment, Accounta- 
bility, and Research has conducted this evaluation. 

Methods 


1 Effective July 1, 2013, this law was repealed and a new statute was written, §18-21. However, 
both the new and old statutes require an evaluation of the implementation of the statewide profes- 
sional development plan, including its effectiveness, efficiency, and impact. 
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Executive Summary 


The following results are based on 1,018 reports submitted by the PD providers using 
an online reporting tool, during three data collection periods that spanned the period from 
June l, 2012 through May 31, 2013. This number of reports represents a large increase from 
the previous year, during which 572 reports were made. In addition to the provider reports 
nearly 6,000 individuals responded to a survey with collection periods in the late fall of 2012 
and spring of 2013. This number of respondents represented a 65.6% response rate. 

Findings and Discussion 

With regard to basic issues of implementation, by far the most notable trend was the 
decrease in participation in the PD Master Plan by the RESAs during this period. Before 
providing details about this decline it should be noted that this trend was reversed in the 
2013-2014 PD Master Plan (an evaluation of which is currently underway), making 2012- 
2013 a low point. This shift will be covered in subsequent evaluation reports; however for the 
3-year time period from 2010-2011 through 2012-2013, 

• the Center for Professional Development (CPD) increased its slate of sessions more 
than fivefold; 

• institutions of higher education (IHEs) with teacher preparation programs held 
steady at a very low level of participation with only one (Marshall University) of 12 
participating; 

• the WVDE more than doubled its participation; and 

• the RESAs reduced their collective contribution to the PD Master Plan by about two 
thirds. 

As for attendance in professional development sessions offered by the four provider groups 
required to participate, the WVDE was responsible for more than three quarters of all partic- 
ipants in PD Master Plan sessions in 2012-2013. 

RESA directors indicated on multiple occasions following the publication of the 
2010-2011 evaluation report that one insurmountable impediment to their participation was 
the schedule they were required to follow in submitting their lists of sessions for inclusion in 
the plan. They argued that because they must provide PD in response to the strategic plans 
of the districts they serve (submitted in early fall), they could not predict at the time the PD 
Master Plan was being developed, what PD they would need to offer. For this reason, the 
Board allowed all providers to update their plans in late fall, beginning in 2012. Marshall 
University and seven WVDE offices took this opportunity to add sessions to their plans; 
none of the RESAs did. 

In 2013-2014, however, RESAs seem to have changed their approach to the PD Mas- 
ter Plan. In that plan the RESAs vary in their number of offerings from a low of two sessions 
(RESA 8) to a high of 49 (RESA 1). The higher numbers of offerings by most RESAs more 
closely reflect their reports for professional development in their annual reports. This is a 
situation that will need continued monitoring, as RESAs take a larger role in providing pro- 
fessional development, and as the State Board works to develop a more coherent statewide 
system for professional learning. 
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Other notable implementation trends include the fact that nearly 7,400 educators in 
the state participated in sessions of 30 hours or more duration, which is the minimum that 
recent reviews of the research identify as producing changes in teacher practice and/or stu- 
dent performance (Yoon, Duncan, Scarloss, & Shapley, 2007). Of the sessions offered during 
the reporting period, about a third were brief, informational sessions, another third were 
half-day to slightly less than two-day technical training sessions and the remaining third 
were sessions of two or more days duration. 

This was the first year that providers were asked to report how aligned their offerings 
were with the new Board standards for professional development, which are an adaptation 
of the Learning Forward standards. Overall, there was less than a 60% level of compliance 
with the standards. The Center for Professional Development reported 100% compliance for 
all standards for all sessions, while Marshall University reported a rate of compliance at 
about 78%, followed by the RESAs at 67% and WVDE at 48%. By their own self-reports pro- 
viders, overall, are strongest with regard to the following Board professional learning stand- 
ards, with which they reported about two-thirds of their sessions aligned: 

1. Occurs within learning communities committed to continuous improvement, collec- 
tive responsibility, and goal alignment. 

5. Integrates theories, research, and models of human learning into learning designs to 
achieve its intended outcomes. 

7. Aligns its outcomes with educator performance and student curriculum standards. 

Weakest alignment (less than half of reported sessions) with the Board professional learning 
standards was reported for the following: 

4. Uses a variety of sources and types of student, educator, and system data to plan, as- 
sess, and evaluate professional learning. 

6. Applies research on change and sustains support for implementation of professional 
learning for long-term change. 

The remaining standards ([2] Requires skillful leadership to develop capacity, advocate, and 
create support systems for professional learning; [3] Requires prioritizing, monitoring, and 
coordinating resources for educator learning) fell in between. 

There seemed to be some confirmation in the participant survey responses for the lack of 
alignment with Standard 6; only two thirds of participants agreed or strongly agreed that the 
professional development session they attended had included "adequate embedded follow- 
up and continuous feedback.” In other words, ongoing follow-up to help them succeed in 
their implementation was lacking in a third of participants' experiences. 

The standards are new for state providers, and although they were included in infor- 
mation that went out to providers during the time the PD Master Plan was developed, it is 
unclear how aware of them most providers are. Data about the Board standards in this re- 
port should be considered baseline, and we will follow trends regarding providers’ alignment 
with them in upcoming evaluation studies. Further, relying primarily on provider self- 
reports to measure alignment with Board standards for professional development is a limita- 
tion that should be noted. 
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Turning now to perceived quality, alignment with Board goals, and impacts, we note 
the following trends: 

• This analysis showed no overall gain in quality since 2011-2012, with a score of 3.9 
on a 5-point scale both years; there was only a slight gain compared with 2010-2011, 
which saw a score of 3.8. 

• With regard to participants’ recognition that the professional development was 
helpful in meeting Board goals for professional development, the overall agreement 
rate of 76.1%, exceeded the previous two years (i.e., 2010-2011, 67.8% and 2011- 
2012, 51.2%) There may be three factors at work in the relative high rate experi- 
enced this year: (a) providers were guided to a greater degree than previous years by 
the Board goals as they planned their sessions; (b) the goals were written more 
broadly, so it was easier for participants to see the connections; and/or (c) providers 
were required to select only one goal as aligned to the offerings in the PD Master 
Plan and were, therefore, less likely to select multiple, less closely tied goals for indi- 
vidual offerings. 

• Although effect sizes ranged from moderate to very large, there was only a slight 
gain for perceived impacts on knowledge, and slight decreases for impacts on prac- 
tice and attitudes/beliefs. 

Taken together, these results show general satisfaction with the professional development 
participants experienced, but do not show much movement in improving the quality and 
impact. Further, the notable improvement in alignment with Board goals may have more to 
do with the goals themselves than with providers’ efforts to align their offerings. 

Limitations 

Implementation findings in this report are based on self-reports by providers, which 
may be subject to bias. Further, the RESAs participated at very minimal levels both in terms 
of the number of sessions they submitted to be included in the 2012-2013 PD Master Plan 
and the numbers of participants they reported in those sessions. Consequently, it is un- 
known if the findings reported here are an accurate portrayal of RESAs offerings more gen- 
erally. Lastly the use of a retrospective pretest/posttest methodology to assess changes in 
knowledge, behavior and skills, and attitudes and beliefs poses some concerns, especially its 
potential to inflate effect sizes. Therefore, we recommend cautious interpretation of our es- 
timates of effect size, as they may be somewhat inflated. 
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Recommendations 

As this report is written, the West Virginia Board of Education has engaged the Na- 
tional Commission on Teaching and America's Future to lead an effort to overhaul the state's 
approach to professional development. Recommendations in previous evaluations of the 
Board's Master Plan for Statewide Professional Development will likely be addressed in the 
course of this overhaul. In the meantime we offer the following recommendations: 

• Find ways to increase the participation of institutions of higher education with teach- 
er preparation programs from the current one IHE (Marshall University) to the full 
12 IHEs that should be a part of it. 

• Consider developing goals for professional development with a longer view, commit 
to those goals for a sustained period of time and publicize them broadly, so that those 
planning for and providing professional development at all levels will be fully aware 
of them and willing to align their efforts to form a more coherent statewide approach. 

• Provide information about the Board standards for professional learning to all pro- 
fessional development providers working in the state, and develop training and in- 
centives that will motivate providers to craft their offerings to meet those standards. 
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Introduction 


West Virginia Code §i8-2-23a 2 required the West Virginia Board of Education 
(WVBE) to establish annual professional development goals for public schools; to coordinate 
professional development programs; and to guide program development, approval and eval- 
uation. The legislative intent of this section of state law was 

(1) To provide for the coordination of professional development programs by the 
State Board; 

(2) To promote high-quality instructional delivery and management practices for a 
thorough and efficient system of schools; and 

(3) To ensure that the expertise and experience of state institutions of higher educa- 
tion with teacher preparation programs are included in developing and implementing 
professional development programs. 

Toward these ends, the WVBE (2012) adopted the following goals for professional 
development for the 2012-2013 school year: 

To provide professional development that— 

1. Aligns with curriculum standards to increase educator effectiveness in the arts, 
world languages, health, physical education, career/technical, reading/English 
language arts, mathematics, science, and social studies. 

2. Focuses on developing in-depth understanding of the essential features of the 
growth and development of the personal, physical, social, and emotional needs of 
each student, including providing students with personalized pathways and guid- 
ance to help them have productive and satisfying lives. 

3. Develops the leadership competencies, professional culture, and characteristics 
necessary to increase the support of teaching and learning. 

West Virginia Code §i8-2-23a further required that, each year, once the annual goals 
are set, the state board must submit the goals to the major state agencies responsible for 
providing professional development to teachers, administrators, and other professional 
education staff statewide, including the West Virginia Department of Education (WVDE), 
the West Virginia Center for Professional Development (CPD), the regional education service 
agencies (RESAs), and the Higher Education Policy Commission (HEPC). These agencies 
then are required to collaborate in the development of an annual master plan for 
professional development aligned with the goals. The law states, 

The Master Plan shall serve as a guide for the delivery of coordinated professional 
staff development programs by the State Department of Education, the Center for 
Professional Development, the state institutions of higher education and the regional 
educational service agencies beginning on the first day of June in the year in which 
the Master Plan was approved through the thirtieth day of May in the following year. 

This section does not prohibit changes in the Master Plan, subject to State Board ap- 
proval, to address staff development needs identified after the Master Plan was ap- 
proved. 


2 Effective July 1, 2013, this law was repealed and a new statute was written, §18-21. However, 
both the new and old statutes require an evaluation of the implementation of the statewide profes- 
sional development plan, including its effectiveness, efficiency, and impact. 
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Additionally, the statute requires evaluation of the effectiveness of the professional 
staff development programs. The WVBE charged the WVDE Office of Research to meet this 
requirement. 

Lastly, although not specifically required by law, the Board chose to adopt standards 
for professional development based on the Learning Forward (formerly National Staff 
Development Council) Standards for Professional Learning, and included them in the 2012- 
2013 Master Plan. According to the standards, professional learning that increases educator 
effectiveness and results for all students— 

• Occurs within learning communities committed to continuous improvement, col- 
lective responsibility, and goal alignment. 

• Requires skillful leadership to develop capacity, advocate, and create support sys- 
tems for professional learning. 

• Requires prioritizing, monitoring, and coordinating resources for educator learn- 
ing. 

• Uses a variety of sources and types of student, educator, and system data to plan, 
assess, and evaluate professional learning. 

• Integrates theories, research, and models of human learning into learning de- 
signs to achieve its intended outcomes. 

• Applies research on change and sustains support for implementation of profes- 
sional learning for long-term change. 

• Aligns its outcomes with educator performance and student curriculum stand- 
ards. 


Goals of the Evaluation 

This evaluation study provides summative information about the implementation of 
the Master Plan for Professional Staff Development for 2012-2013 as follows: 

• Implementation of planned sessions, including the number of teachers, adminis- 
trators, and others who participated in the professional development sessions 
targeted at each of the goals listed in the PD Master Plan from June 1, 2012 
through May 31, 2013; sessions planned versus sessions delivered; adherence to 
the Board standards for professional development duration of the sessions; their 
location; attendance at sessions conducted by each of the providers; and the de- 
livery mode (i.e., online, face-to-face, blended, or other). 

• Participant perceptions about the sessions’ adherence to research-based prac- 
tices for high quality professional development, including whether sessions were 
(a) intensive in nature; (b) specific and content-focused; (c) relevant to partici- 
pants’ current needs and professional circumstances; (d) hands-on with active 
learning opportunities; (e) supported by follow-up discussion or collaboration at 
participants’ workplaces or online; (f) supported by related follow-up PD ses- 
sions; and (g) beneficial and had a positive impact on participants’ students 
and/or schools. 

• Participant perceptions about the sessions’ helpfulness with regard to reaching 
the specific goals for professional development as specified in the 2012-2013 PD 
Master Plan 

• Participants’ perceived (self-reported) outcomes resulting from their involve- 
ment in professional development associated with the PD Master Plan— for ex- 


2 / Implementation of the Master Plan for Statewide Professional Staff Development for 2012-2013 


Methods 


ample, changes in educators’ (a) knowledge; (b) behaviors and skills; and (c) atti- 
tudes and beliefs. 

Methods 

Population to be Studied 

This study examines the performance of professional development providers in im- 
plementing the 2012-2013 Master Plan for Professional Staff Development (PD Master 
Plan), which was approved by the West Virginia Board of Education in May 2012. Providers 
in the list included the Center for Professional Development, two centers from Marshall 
University (the only institution of higher education [IHE] that participated in the plan), all 
eight RESAs, and 14 offices from the West Virginia Department of Education (WVDE)— a 
total of 25 providers in all. 

Unlike previous years, in 2012-2013 all WVDE offices involved in teacher prepara- 
tion, curriculum and instruction, or school leadership participated in the plan— an increase 
from eight offices in 2011-2012 to 14 offices in 2012-2013. Similar to previous years, partic- 
ipation by public IHEs with teacher preparation programs were largely absent from partici- 
pation in the PD Master Plan. Only Marshall University participated. 

Sampling Procedures 

All 25 professional development providers in the PD Master Plan reported on ses- 
sions they conducted as part of the Plan, providing the (a) title of session, (b) alignment of 
the session with the Board standards for professional development, (c) beginning and end- 
ing dates, (d) duration of the session in hours, (e) format of the sessions (i.e., face-to-face, 
online, or blended), (f) number of participants, and (g) e-mail addresses for all participants. 

Using the e-mail addresses of participants reported by the providers as attending 
sessions held from June 1, 2012 through March 31, 2013, we conducted two online surveys of 
teachers, administrators, and others who attended the professional development. For both 
the first and second participant surveys (conducted in late fall, November-December, 2012 
and spring, April-May, 2013), we applied multistage sampling— systematic, stratified, and 
simple random— to select participants for this study, using the following procedure: 

We combined the e-mail addresses— each e-mail address with its associated PD Mas- 
ter Plan session ID and provider— into one comprehensive Excel file ( N = 6,528 for the first 
participant survey; N = 13,122 for the second). 

• Participants were sorted by e-mail address and assigned a random number. The 
sample was then resorted by random number and the first occurrence of each in- 
dividual’s e-mail was selected. For the spring survey, an extra step was involved 
to avoid contacting any individual twice in one year. The sample was checked 
against the sample from the fall, and any case that had been previously surveyed 
was removed. 

• The sample was then stratified by provider and a simple random sample was 
drawn for each provider. 

Overall, sampling for each provider, inclusive of both survey periods, is shown in Ta- 
ble 1 (below). 
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Table 1. Total Attendance Reported, E-mail Addresses Provided, and Sample Selected for Participant 


Survey 


Attendance 

reported 

Email addresses 
provided 

Sample 

selected 

Provider 

All providers 

32,582 

19,650 

9,129 

Provider groups 

Center for Professional Development 

2,818 

2,665 

994 

Public institutions of higher education (Marshall) 

839 

743 

468 

Regional educational service agencies (RESAs) 

4,090 

2,049 

1,160 

West Virginia Department of Education (WVDE) 

24,835 

14,193 

6,507 

Individual providers 

Center for Professional Development* 

2,818 

2,665 

994 

Marshall University Clinical Experiences and Professional 

120 

200 

134 

Development Schools 




Marshall University June Harless Center 

719 

543 

334 

RESA 1 

912 

500 

211 

RESA2 

259 

165 

91 

RESA 3 

566 

73 

49 

RESA 4 

460 

197 

99 

RESA 5 

200 

173 

96 

RESA 6 

708 

265 

159 

RESA 7 

798 

516 

318 

RESA 8 

187 

160 

137 

WVDE Office of Assessment and Accountability 

1,130 

693 

478 

WVDE Office of Career and Technical Accountability and 




Support 

1,268 

412 

295 

WVDE Office of Career and Technical Innovations 

115 

43 

43 

WVDE Office of Career and Technical Instruction 

1,587 

404 

324 

WVDE Office of Child Nutrition 

70 

51 

51 

WVDE Office of Early Learning 

2,624 

2,650 

718 

WVDE Office of Federal Programs 

62 

34 

26 

WVDE Office of Healthy Schools 

369 

331 

289 

WVDE Office of Instruction 

1,577 

1,070 

666 

WVDE Office of Instructional Technology 

6,802 

2,567 

943 

WVDE Office of Optional Educational Pathways 

596 

463 

354 

WVDE Office of Professional Preparation 

2,166 

1,672 

766 

WVDE Office of School Improvement 

1,244 

1,110 

508 

WVDE Office of Special Programs 

5,225 

2,693 

1,046 


*The provider groups are specified in West Virginia Code §18-2-23a. Because the Center for Professional 
Development is a single entity, it appears in both the provider groups and individual providers lists above. 

It should be noted that participants in professional development scheduled during 
the months of April and May, 2013 were not surveyed to avoid interfering with the 
WESTEST 2 testing window and in recognition of the fact that teachers and others are 
difficult to reach with the onset of the summer break. 
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Sample Size, Power, and Precision 

Knowing the population of each provider, we used sample size calculation software 3 
to determine what sample size was needed to attain a 95% confidence level with a +/-3%, 
margin of error, and then drew samples sufficient to achieve that level of confidence for each 
provider with a 70% response rate. The sample amounted to about 46.5% of the e-mail ad- 
dresses submitted by providers, and included more than 9,000 attendee e-mail addresses. 

Measures and Covariates 

As mentioned above, providers used an online SurveyMonkey tool to report essential 
information about each professional development session they conducted, including (a) 
name of provider, (b) contact information, (c) title of session, (d) alignment with Board 
goals, (e) duration in hours, (f) beginning and ending dates, (g) county location, (h) format 
(face-to-face, online, or blended), (i) number of participants, (j) e-mail addresses for all par- 
ticipants, and (k) comments (optional). 

Information collected using this session report was combined with information about 
the planned sessions in the PD Master Plan, which allowed us to report on sessions held re- 
lated to each of the West Virginia Board of Education (WVBE) goals for professional devel- 
opment, and other information about implementation of the plan. 

To collect participants’ perceptions about the quality, relevance, and effectiveness of 
the training, an online survey questionnaire posted via SurveyMonkey, the WV PD Master 
Plan: 2012-2013 Participant Survey, was used. Each participant in the survey was contacted 
up to five times, about only one PD session they attended between June 1, 2012 and March 
31, 2013. Responses about these individual provider offerings were then aggregated to pro- 
vide overall perceptions about various aspects of the training offered by each provider. The 
questionnaire included a section on participant demographics and three sections on partici- 
pant perceptions about the PD they attended. 

Independent variables related to participants included (a) county, (b) programmatic 
level, (c) professional role, and for teachers, (d) main content area taught. Dependent varia- 
bles were participant perceptions about various aspects of the PD sessions, including (a) the 
sessions’ adherence to research-based practices for high quality professional development; 
(b) the sessions’ helpfulness with regard to the specific Board goals for professional devel- 
opment; and (c) perceived (self-reported) outcomes of participants’ involvement in the pro- 
fessional development. 

Lastly, we surveyed providers in early June 2013, to discover the reasons why some 
sessions listed in the PD Master Plan were not offered. To collect these data, we sent email 
messages to the executive directors of each provider organization which listed the sessions in 
the PD Master Plan for which we had not received any reports, and asked them to supply the 
main reason the session was not offered. 


3 MaCorr Research (n.d.) Sample Size Calculator. Available online at 
http : / / www.macorr.com /sample-size-calculator .htm. 
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Research Design 

We used a multimethod research design for this project, and used descriptive statis- 
tics to explore five distinct areas: (a) implementation of the professional development ses- 
sions listed in the PD Master Plan, (b) description of participants, (c) participants’ 
perceptions of the quality of professional development, (d) participant perceptions of the 
extent to which professional development met the goals established as part of the PD Master 
Plan, and (e) participant perceptions about the impact of the professional development on 
their knowledge, behaviors, and attitudes/beliefs. We also conducted a variety of post -hoc 
exploratory analyses to determine level of participation in the PD Master Plan by the various 
provider groups, which by law, are required to participate in the formation of the plan and 
its evaluation. Each of these investigations involved a variety of analyses, as described below. 

Description of professional development 

We used descriptive analyses including frequency distributions and cross-tabulations 
to provide an overview of the professional development offered by each provider during the 
2012-2013 academic year and trends from 2010-2011 to 2012-2013. This analysis included a 
description of which events were published in the Master Plan and then provided, as well as 
those that were published, but never provided (e.g., canceled events). We analyzed results of 
the missing sessions reports submitted by providers that did not deliver all of the sessions in 
their plan. 

Description of participants 

We conducted additional frequency analyses to examine the composition of the par- 
ticipant survey sample with respect to key demographic variables noted above (see Measures 
and Covariates). 

Participant perceptions about the extent to which the professional development used 
research-based practices 

We calculated average ratings and frequency distributions to describe the extent to 
which participants described the PD session as adhering to research-based practices for high 
quality professional development, that is, whether the professional development was (a) in- 
tensive in nature, (b) specific and content-focused, (c) relevant to their needs as educators, 
(d) hands-on including active learning opportunities, (e) supported by follow-up discussion 
or collaboration at their school, office, or online, (f) supported by follow-up professional de- 
velopment sessions, and (g) beneficial and positive for students and/or schools. These re- 
sults are presented for the overall sample as well as disaggregated by provider, provider 
group, and the duration of the training. 

Participant perceptions about the extent to which the professional development met the 
goals established by the WVBE as part of the 2011-2012 PD Master Plan 

We used descriptive statistics (i.e., mean and frequency distribution) to examine the 
extent to which the professional development provided in 2012-2013 met the goals set forth 
by the WVBE as part of the 20112-2013 PD Master Plan. To accomplish this, we first select- 
ed all response records in the data set involving respondents who attended a professional 
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development event that providers indicated was aligned to, for example, Goal l as listed in 
the PD Master Plan. Then we determined respondents’ ratings regarding the extent to which 
the event met this specific goal, and reported the percentage of total respondents who indi- 
cated they agreed or strongly agreed that the professional development was helpful in meet- 
ing Goal 1. We repeated this procedure for Goals 2 and 3. 

Participant perceptions about the impact of the professional development on their 
knowledge, behaviors, and attitudes/beliefs 

The participant survey includes three pairs of items designed to assess the impact of 
the professional development experience upon participants’ knowledge, behaviors, and atti- 
tudes/beliefs. Each pair consists of an item that asks respondents to rate their knowledge, 
behaviors, or attitudes/beliefs before participating in the professional development and then 
provide ratings for after having participated in professional development. 

We used a retrospective pretest/posttest design to determine if respondents’ posttest 
ratings are significantly different from pretest ratings (i.e., paired samples t tests). In addi- 
tion, we conducted analyses of the effect size for the difference in respondents’ pre-/posttest 
ratings to determine whether any statistically significant differences also have practical sig- 
nificance. Results were examined for the entire sample and disaggregated by provider, pro- 
vider group, and duration of the training. 

Results 

Results in this section are presented in three major sections: the first is devoted to 
implementation of the plan, based on provider reports; the second focused on the quality, 
alignment with Board goals, and perceived effectiveness of the professional development 
sessions, based on a survey of participants; and the third includes ad hoc analyses of other 
qualitative data. 

Implementation of the PD Master Plan: Analysis of Provider Reports 

The following results are based on 1,018 reports submitted by the PD providers using 
an online reporting tool, during three data collection periods: November (covering June 1— 
October 31, 2012), April (covering November 1, 2012-March 31, 2013), and June (covering 
April l-May 31, 2013). This number of reports represents a large increase from the previous 
year, during which 572 reports were made. 
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Level of implementation 

The number of different professional development session titles offered by providers 
ranged from one (WVDE Office 
of Child Nutrition) to 85 (WVDE 
Office of Instructional Technolo- 
gy). The RESAs each submitted 
three session titles with, in each 
case, one session aligned with 
each of the Board’s goals. In pre- 
vious years, RESAs coordinated 
their session title submissions, 
all eight of them submitting the 
same set of titles (i.e., eight in 
2011-2012 and seven in 2012- 
2013). This is a notable reduc- 
tion in participation by the RE- 
SAs, during a time when other 
provider groups have increased 
their participation, especially 
CPD and offices in the WVDE. 

Figure 1 shows the overall trends 
for participation in the PD Mas- 
ter Plan— that is, the number of 
session titles each provider cate- 
gory submitted to be included in 
the plan. 

Overall, 81.2% of the professional development sessions included in the final 2012- 
2013 PD Master Plan (post addendum period in November 2013) were actually provided to 
educators across the state (Figure 2). This percentage is up slightly from 77. 5 % the previous 
year. PD providers were asked to report dates, locations, duration, alignment with the Board 
standards for professional development, attendance figures, as well as attendee e-mail ad- 
dresses for all sessions they included in the PD Master Plan. If we received none of this in- 
formation for a particular session, we counted that session as not provided or reported. In 
some cases, an individual session listed in the PD Master Plan was held several times with 
different groups of educators in various locations during the course of the academic year. In 
those cases, we aggregated the e-mail addresses and attendance numbers and reported them 
as one of the planned sessions listed in the PD Master Plan (PD provided column), and also 
broke out the number of individual repetitions of the sessions (repetitions) held (see Table 4 
in the Appendix, page 29. 



■CPD 

■Higher Ed 
■RESAs 
■ WVDE 


Figure 1. Number of Session Titles Submitted by Provider 

Category 

Participation in the PD Master Plan by Offices in the West Viginia 
Department of Education rose dramatically in 2012-2013 while 
participation rose steadily for the Center for Professional 
Development, remained stable for institutions of higher 
education, and declined for regional education service agencies. 
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Figure 2 shows the level at which each of the providers followed the plan they sub- 
mitted and had approved as part of the PD Master Plan (see the figure caption for a brief 
discussion). For details about each of the providers, see Table 4 in the Appendix (page 29). 
For session topics not reported as delivered, providers were asked to submit the main rea- 
son, in each case, why they did not hold any sessions under those topics. The most prevalent 
reasons included the following (next page): 
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Figure 2. Percentage of Professional Development Included In the PD Master Plan That Were Provided During 
2012-2013 

Most providers were able to provide all or nearly all of the sessions they submitted for the PD Master Plan 
(including the addendum period in the fall of 2012). The remainder provided more than half of their planned 
sessions with one notable exception, the WVDE Office of Federal Programs. This office was affected by a 
WVDE reorganization that resulted in staff being reassigned from other offices to this office, with changed 
priorities. Other WVDE Offices were affected to a lesser extent by the shift in priorities resulting from the 
Board's response to the 2012 Education Audit. 


Implementation of the Master Plan for Statewide Professional Staff Development for 2012-2013 / 9 


Results 


• The session topic was combined with another, often in collaboration with a RESA 
or another WVDE office (19 responses). 

• State Board or Department priorities or policies changed (17 responses). 

• There was a lack of requests or sufficient participant registrations (12 responses). 

• The session was scheduled just before or after the reporting year (11 responses). 

Attendance trends 

Overall attendance trends paralleled 
providers’ levels of participation in the PD 
Master Plan. Attendance at WVDE- and 
CPD-sponsored sessions rose from the pre- 
vious years, while attendance at IHE- and 
RESA-sponsored events fell. Overall, attend- 
ance was up by about 50% (Table 2). Attend- 
ance at WVDE-sponsored professional 
development accounted for over three quar- 
ters of the total attendance at sessions included in the PD Master Plan. 

Format, duration, and time span 

By far, the prevailing ex- 
perience for participants in pro- 
fessional development was to 
meet face-to-face. More than 
three quarters of participants 
were in face-to-face sessions, 
followed by 15% in online expe- 
riences, which were offered pri- 
marily by the WVDE Office of 
Instructional Technology. The 
remaining participants— about 
696 — were in sessions that 
blended online and face-to-face 
experiences (Figure 3). 

Overall, professional de- 
velopment sessions provided 
through the PD Master Plan 
were about evenly divided 
among information sessions (up to 4 hours), technical training (5-13 hours), and sustained 
professional development sessions (14 or more hours). For details about each of the provid- 
ers, see Table 5 in the Appendix (page 30). 

Looking at numbers of participants in these sessions, however, provides a slightly 
different story (see Table 6 in the Appendix, page 31). The categories in descending order of 



Face-to-face 
1 Online 
Blended 


WVDE 


0 % 20 % 40 % 60 % 80 % 100 % 

Figure 3. Percentage of Attendance in Face-to-Face Session Versus 
Other Formats, Overall and by Provider Category 
Of the four provider categories, the West Virginia Department of 
Education was the source of most online professional 
development experiences for educators in 2012-2013. 


Table 2. Attendance Trends by Provider 
Category, 2011-2012 and 2012-2013 


Number of attendees 

Provider category 

2011-2012 

2012-2013 

Total 

21,552 

32,582 

CPD 

1,109 

2,818 

Higher education 

1,181 

839 

RESAs 

4,657 

4,090 

WVDE 

14,605 

24,835 
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participation were sustained (38%), technical training (35%), and informational (28%). Four 
providers took the lead in providing sustained professional development to the greatest 
numbers of participants. They were, in descending order, the WVDE Office of Instructional 
Technology, the WVDE Office of Special Programs, the Center for Professional Develop- 
ment, and the WVDE Office of Instruction. 

Recent research reviews, however, indicate that many more contact hours than 14 are 
needed to positively affect student achievement (Blank, de las Alas, & Smith, 2008; Clewell, 
Campbell, & Perlman, 2004), with 30 hours being considered the minimum (Yoon, Duncan, 
Lee, Scarloss, & Shapley, 2007). In light of these findings and for the purposes of this report, 
we devised a new category for duration, a recommended category of 30 or more contact 
hours. The WVDE Office of Instructional Technology far exceeded any other providers in the 
number of participants in sessions of 30 hours or longer duration, with 3,068 participants. 
Other providers offering relatively large numbers of participants in professional develop- 
ment at the recommended level of contact hours included the Center for Professional Devel- 
opment (1,323), The WVDE Office of Instruction (984), and the WVDE Office of Special 
Programs (864). Altogether, nearly 7,400 educators in the state participated in sessions of 
30 hours or more duration (Table 6 in the Appendix). For a visual display of these data, see 
Figure 4 below. 

On the other hand, half or more of some providers' offerings fell within the informa- 
tional category— that is, having a duration of 4 hours or less. These providers included RESA 
1 (56.1%), RESA 3 (69.2), RESA 6 (62.5%), WVDE Office of Assessment and Accountability 
(54.2%), WVDE Office of Career and Technical Instruction (71.6%), WVDE Office of Federal 
Programs (83.3%), and the WVDE Office of Professional Preparation (55.6%; Table 5). 

As for timespan, the overall average timespan was 21.3 hours. The providers with the 
longest average timespans were Marshall University's Clinical Experiences and Professional 
Development Schools (146.3 days), Marshall University's June Harless Center (103.1 days), 
and RESA 4 (127.5 days; Table 6). The Marshall University June Harless Center and RESA 4 
also had high percentages of sessions that had durations of at least 14 hours (sustained and 
recommended categories combined), with about 61% and 68% respectively of their sessions 
falling into those categories (Table 5). 

Location of offerings 

Professional development sessions offered by providers in the PD Master Plan were 
held in every county except Calhoun, Pendleton, and Tyler; however, results of the Partici- 
pant Survey revealed that there were participants in PD Master Plan sessions from all coun- 
ties (see next section). More than 4,300 participants took part in sessions held online, for 
which no county location was designated. Attendance was most concentrated in Kanawha 
(7,882 participants), Monongalia (2,675), Harrison (1,758), and Raleigh (1,756) counties. All 
in all, sessions were well dispersed across the state. 

Adherence to Board Standards for Professional Development 
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For the 2012-2013 PD Master Plan, the State Board adopted the Learning Forward 
standards for professional development. As a baseline measure of the extent to which pro- 
viders have adopted those standards, we included questions in the providers’ online report- 
ing form based on each of the Learning Forward standards. 



Number of participants 


Informational (up to 4 hours) Technical (5-13 hours) ■ Sustained (14-29 hours) ■ Recommended (30 hours or more) 

Figure 4. Number of Participants in Professional Development by Duration by Provider 

Relatively large numbers of participants took part in professional development that included 14 or more 
contact hours, depicted here in orange and brown. The WVDE Office of Instructional Technology led in 
providing professional development at the recommended level of 30 hours or more to the greatest 
number of participants (more than 3,000). Still overall, the majority of participants (62%) attended 
professional development sessions lasting 13 hours or less (shown in blue). See Table 6 in the Appendix 
for details. 
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The Board calls for professional development experiences that meet the following 
standards: 


1. Occurs within learning communities committed to continuous improvement, collective re- 
sponsibility, and goal alignment. 

2. Requires skillful leadership to develop capacity, advocate, and create support systems for pro- 
fession learning. 


3. Requires prioritizing, monitoring, and coordinating resources for educator learning. 

4. Uses a variety of sources and types of student, educator, and system data to plan, assess, and 
evaluate professional learning. 

5. Integrates theo- 
ries, research, 
and models of 
human learning 
into learning de- 
signs to achieve 
its intended out- 
comes. 

6. Applies research 
on change and 
sustains support 
for implementa- 
tion of profes- 
sional learning 
for long-term 
change. 

7. Aligns its out- 
comes with edu- 
cator perfor- 
mance and stu- 
dent curriculum 
standards. 

Figure 5 displays the 
variation among pro- 
vider categories, with 
WVDE reporting ad- 
herence to Board 
standards for a third 
to just over half of ses- 
sions. On the other 
hand, CPD reported 
that all of their ses- 
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Figure 5. Adherence of Professional Development Sessions to Board Standards as 

Reported by Providers , by Provider Category 
Overall, the West Virginia Department of Education (WVDE) reported the lowest 
level of adherence to Board standards followed in ascending order by the 
regional education service agencies (RESAs), institutions of higher education 
(Higher Ed), and the Center for Professional Development (CPD). CPD reported 
that all of their sessions met all seven standards. 


sions met all seven 
standards. The other 
provider groups, 
ranked between these 
two, with the higher 
education providers 
(i.e., two centers at 

Marshall University) reporting more frequent adherence to the standards than RESAs. 
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Overall, Standard l was most often cited as being met (66.4%), followed in descending order 
by Standard 7 (64.8%), Standard 5 (64.3%), Standard 3 (58.5%), Standard 2 (56.1%), Stand- 
ard 6 (47.3%), and Standard 4 (45.8%). 


For one last look at this implementation issue, Figure 6 displays the overall level of 
adherence to Board standards, as reported by each of the providers. There was much greater 
variation across providers than there was across the standards. A reminder is warranted 
here: These are self-reported data, and as such, may need to be confirmed in future evalua- 
tions with other measures— such as questions on the participant survey. 
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Figure 6. Adherence to Board Standards for Professional Development by Provider 
Providers were asked to report for each session they conducted, which of the seven Board 
standards that session adhered to. Percentages of reported sessions were calculated for each 
of the standards, for each provider, and then a mean percent across all standards was 
calculated for each provider. Those are the values shown here. 
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Analysis of Participant Survey Responses 

The remainder of the Results section is based on data collected via an online survey 
of PD participants who attended professional development sessions held from June l, 2012 
to March 31, 2013. The survey was conducted in two phases: late November through late De- 
cember 2012, to cover professional development provided during the summer and early fall 
months, and mid-April through late May 2013 to cover professional development offered 
during late fall through March. Results here were aggregated from both data collection peri- 
ods. 

The survey random sample was made up of an unduplicated list of 9,129 participants, 
who were asked about one professional development event they attended (see Table 1, page 
4, for details about the breakdown of the sample by provider and provider category). Of this 
sample, 619 were eliminated due to attrition (including bad e-mail addresses, and individu- 
als who contacted us to report that they did not attend the event we asked them about or 
they attended as a facilitator or in some other nonparticipant capacity). After adjusting for 
attrition, the viable sample was reduced to 8,510; of these, we received responses from 
6,360. After removing unusable responses, the dataset was reduced to 5,992 responses. This 
number represents a response rate of 65.6% for the full sample, or 70.4% for the sample ad- 
justed for attrition. 

Demographic characteristics of survey respondents . 

Frequency analyses revealed characteristics of the respondents with respect to key 
demographic variables. 

All PK-12 programmatic 
levels were well represented 
among the respondents to the 
survey (Figure 7). 

Over half of the survey re- 
spondents (59%) were classroom 
or special education teachers, 
with administrators (including 
district central office staff and 
principals) coming in second 
(19%). The rest of the respondents 
occupied a variety of roles, as 
shown in Table 9 (page 34). 

All 57 school districts (in- 
cluding Institutional Education Programs and the West Virginia Schools for the Deaf and the 
Blind) were represented among the respondents, as well as individuals from institutions of 
higher education, RESAs, the WVDE, and others, including about 2 dozen from out of state 
(Table 10, page 35 in the Appendix). 


Higher 

education 

3% 


childhood 



Figure 7. Percent of Respondents at Each Programmatic Level 
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Adherence to research-based practices 

Prior to the Board’s adoption of the Learning Forward standards— which, among 
other things, outline roles, responsibilities, and contextual issues important when conduct- 
ing professional development— this evaluation has focused on design elements of individual 
professional development sessions. We have referred to these elements as research-based 
practices for high quality professional development (see box). Survey respondents were 
asked to respond to seven items about the extent to which the professional development 
event they attended adhered to these practices. Respondents were instructed to respond to 
each statement using a 5-point Likert-type 
response format, that is, 1 ( strongly disa- 
gree), 2 (disagree), 3 (neutral), 4 (agree), 5 
(strongly agree). Before examining the re- 
sults, it should be noted that the response 
format used for these items is most easily 
interpreted by examining the rate of agree- 
ment (i.e., agree or strongly agree) among 
respondents that the seven research-based 
practices were present, versus responses 
indicating disagreement (i.e., disagree or 
strongly disagree) or neutrality about the 
issue. Additionally we calculated a mean 
rate of agreement across the seven quality 
indicators to get an overall quality measure. 

For full results by individual indicator and provider, see Table 11 in the Appendix (page 36). 

Looking at provider groups, overall, the Center for Professional Development 
received the highest levels of agreement (80.9%) that their sessions possessed research- 
based practices for high-quality professional development (Figure 8), followed closely by the 
other provider groups, which each received this endorsement from more than three quarters 
of respondents. In contrast with the analysis of adherence to Board standards, these ratings 
were received from representative samples of participants, not the providers themselves. 
Once again, there was a wide range of agreement when looking at individual providers, 
although only about a third of providers fell below the 75% level of agreement. 

Our final disaggregation compared the ratings for sessions of different duration, in- 
cluding informational (up to 4 hours), technical training (5 to 13 hours), and sustained pro- 
fessional development (14 or more hours). Figure 9 (page 18) and Table 12 (page 37) show 
the results of these analyses. Overall, sustained professional development sessions received 
higher rates of agreement about their adherence to research-based practices (79.0%) than 
either informational or technical training sessions, which were nearly the same at 71.6% and 
71.5% respectively. Practices for which sustained professional development sessions received 
notably higher ratings than the other two formats included intensive, skill practice and 
feedback, content focused, opportunities for collaboration, and embedded follow-up and 
feedback. 


Research-Based Practices for High Quality 

Professional Development 

• Was intensive in nature. 

• Included adequate opportunities to practice 
new skills and receive feedback. 

• Was adequately focused on core content 
knowledge. 

• Was adequately tied to school and district goals 
for student learning. 

• Included adequate opportunities for 
collaboration. 

• Included adequate embedded follow-up and 
continuous feedback. 

• Was beneficial and had a positive impact on our 
students and/or school, overall. 
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Figure 8. Mean Percentage of Respondents Who Agreed or Strongly Agreed That the Session 
They Attended Used Research-Based Practices for High Quality Professional 
Development by Provider Group and Individual Provider 
Survey participants were asked to indicate the extent to which they agreed with statements 
that the professional development was intensive, had active learning, was content focused, 
was aligned to school goals, included collaboration, had embedded follow-up, and was benefi- 
cial overall. The bars in this graph show the aggregated rate of agreement across these seven 
measures, for provider groups and individual providers. 
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Figure 9. Mean Percentage of Respondents Who Agreed or Strongly Agreed That the 


Session They Attended Used Research-Based Practices for High Quality 
Professional Development by Duration of Session 
The bars in this graph show the aggregated rate of agreement across these seven 


measures for research-based practice, for different durations of sessions: 


informational (up to 4 hours), technical training (5 to 13 hours), and sustained 
professional development (14 or more hours). 


In a separate analysis, we calculated the mean score across the seven indicators, us- 
ing the 5-point Likert-type response format, that is, l ( strongly disagree ), 2 (disagree), 3 
(neutral), 4 (agree), 5, to derive a Quality Index score that could give us a sense of the over- 
all quality, and compared this score to the previous two years. This analysis showed no gain 
in quality since 2011-2012, with a score of 3.9 both years; there was only a slight gain com- 
pared with 2010-2011, which saw a score of 3.8. 


Perceived effectiveness in meeting Board goals for professional development 

Each professional development session included in the 2012-2013 PD Master Plan 
was determined by providers to be aligned primarily to one of the Board’s three goals for 
professional development; therefore, we sought to determine the extent to which each of the 
participant’s professional development experience had helped them to realize the goal area 
aligned with the session each one attended. 
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We used descriptive statistical analyses to examine responses for all events associat- 
ed with each goal area. First we disaggregated responses into four datasets: 

• One associated with sessions aligned to Goal l (i.e., “Aligns with curriculum 
standards to increase educator effectiveness in the [content areas]); 

• two associated with Goal 2 (i.e., [2A] “Focuses on developing in-depth under- 
standing of the essential features of the growth and development of the personal, 
physical, social, and emotional needs of each student,” and [2B] “Providing stu- 
dents with personalized pathways and guidance to help them have productive 
and satisfying lives.”); and 

• one associated with Goal 3 (i.e., “Develops the leadership competencies, profes- 
sional culture, and characteristics necessary to increase the support of teaching 
and learning”). 

We then analyzed participants’ responses for each goal area independently. 
Respondents were instructed to respond to statements about the professional development 
using a 5-point Likert-type response format as follows: 1 ( strongly disagree), 2 (disagree), 3 
(neutral), 4 (agree), 5 (strongly agree). A sixth category, not applicable, was included, and 
tallied along with the other responses as an indication of the lack of alignment with the goal 
in question— that is, if the respondent considered the goal in question as not applicable to 
the session he or she attended, we counted this response as a lack of agreement that the 
session was helpful in meeting the goal. The full results for each of the providers appear in 
Table 13 in the Appendix (page 38), organized by goal. A breakdown by provider group and 
overall is found in Figure 10. 

For Goal 1, RESAs were the provider group that received the highest rate of agree- 
ment among participants that the session attended had been helpful, followed closely by 
CPD. Among individual providers the top quartile, with a median of 95.9%, included in de- 
scending order RESAs 2, 7, 4, and 8; while the bottom quartile (median 61.6%) included in 
descending order RESA 5, the WVDE Office of Assessment and Accountability, the WVDE 
Office of Career and Technical Accountability and Support, and RESA 6 (See Table 13 in the 
Appendix, page 38). 

For both parts of Goal 2 (A and B), the IHEs (Marshall University) had the highest 
rate of agreement among the provider groups that the session attended had been helpful. 
Among individual providers the top quartile (median 86.6%) for Goal 2A included in de- 
scending order RESA 4, RESA 5, Marshall University June Harless Center, and the WVDE 
Office of Healthy Schools; the bottom quartile (median 50.6%) in descending order were 
WVDE Office of Career/Technical Accountability and Support, RESA 3, WVDE Office of 
Child Nutrition, and WVDE Office of Early Learning. (See Table 13 in the Appendix, page 
38 ) 

Among individual providers for Goal 2B, the top quartile (median 85.4%) included in 
descending order, RESA 4, Marshall University June Harless Center, WVDE Office of In- 
structional Technology, and the WVDE Office of Healthy Schools; the bottom quartile (me- 
dian 51.9%) in descending order included RESA 8, RESA 7, WVDE Office of Early Learning, 
and the WVDE Office of Child Nutrition (See Table 13 in the Appendix, page 38). 
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Figure 10. Participant Perceptions About Flelpfulness of Session in Meeting Designated Board Goals 

Overall greater percentages of participants— more than 80%— agreed the sessions they attended were 
helpful with regard to Goals 1 and 3 than with either part of Goal 2. Yet, even for Goal 2 more than two 
thirds agreed. 


Finally, for Goal 3, CPD had the highest rate of agreement among provider groups. 
Among individual providers the top quartile (median 92.3%) included in descending order 
CPD, RESA 2, and Marshal University Clinical Experiences and PD Schools; the bottom 
quartile (median 73.9%) in descending order were WVDE Office of Special Programs, RESA 
1, and the WVDE Office of Instructional Technology (See Table 13 in the Appendix, page 38). 

With an overall agreement rate of 76.1%, 2012-2013 exceeded the previous two years 
(i.e., 2010-2011, 67.8% and 2011-2012, 51.2%) with regard to participants’ recognition that 
the professional development was helpful in meeting Board goals. 

Perceived impact of professional development 

The survey contained three pairs of items that asked respondents to use a 4-point 
Likert-type scale (1 [ not at all], 2 [to a small extent], 3 [to a moderate extent], 4 [to a great 
extent]), to rate the extent to which they agreed with statements about themselves both be- 
fore and after having participated in the professional development session they attended, as 
follows: 

Pair 1. Before participating in this PD, to what extent were you knowledgeable 
about the topic it covered? 
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After participating in this PD, to what extent are you knowledgeable about 
the topic it covered? 

Pair 2. Before participating in this PD, to what extent did you practice behaviors or 
skills it taught? 

After participating in this PD, to what extent do you practice behaviors or 
skills it taught? 

Pair 3. Before participating in this PD, to what extent did you hold attitudes/beliefs 
it encouraged? 

After participating in this PD, to what extent do you hold attitudes/beliefs it 
encouraged? 

A fifth response category was included, but only used to allow respondents to indicate the 
item was not applicable to them. These responses were not used when calculating mean 
scores. 


We used a retrospective pretest/posttest design to assess the extent to which survey 
respondents perceived a change in their own knowledge, behaviors, and beliefs and attitudes 
as a result of participating in professional development. A series of paired-samples t tests 
were conducted using respondents’ pre- and post-ratings. These analyses tested for statisti- 
cally significant differences between respondents’ pre- and post-ratings, with time as the in- 
dependent variable. When statistically significant differences were found (i.e., p <.05), it is 
reasonable to say that the difference observed between participants’ pre- and posttest results 
are not likely to be due to chance. That is, there is some systematic reason underlying the 
difference. This analysis does not allow one to infer a cause for the difference. It merely de- 
scribes the presence of a significant difference. 

One limitation of significance testing is that it tells us very little about the magnitude 
of any observed differences. We detect a difference, but cannot tell from the t test if the dif- 
ference is meaningful in a practical sense. Calculating an effect size is one way to explain the 
magnitude of any statistically significant differ- 
ences. In this study, we used Cohen’s d as a 
measure of effect size. This statistic is commonly 
used in simple pretest/posttest designs, although 
its interpretation is often debated in social sci- 
ences (see the Limitations of the Study section, 
page 27, for more about this debate). The guide- 
lines we used for interpreting the meaning of the 
effect sizes in this study are found in Table 3. Paired-samples t tests were conducted for 
three impact items: (1) knowledge about the topic of professional development, (2) use of 
behaviors and skills related to the topic, and (3) presence of attitudes/beliefs advocated by 
the professional development. 


Table 3. Interpretation of Effect Size 


Estimates Used in this Study 


Value for Cohen's d 

Interpretation 

Less than .4 

Small effect 

.4 to .7 

Moderate effect 

.8 or 1.1 

Large effect 

1.2 and above 

Very large effect 
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Aggregated mean presession scores hovered 
around 3, indicating that participants, overall, 
thought they had a moderate level of knowledge, 
skill, and attitude/belief prior to engaging in the 
session. They assessed themselves at the midpoint 
between moderate and great levels after the session, 
indicating that participants, overall thought they 
had grown professionally as a result of the experi- 
ence. 

Significance testing revealed that the results 
were significant at the p <.05 level for all but one of 
the 90 tests we ran— and the great majority of those 
tests were statistically significant at the p < .001 
level (see Table 14 in the Appendix, page 40) 3 

Aggregating all results, respondents per- 
ceived a very large impact on the extent of their 
knowledge as a result of attending the session, with 
a large impact on their practice and moderate im- 
pact on their attitudes and beliefs (Figure 
12). Similarly, across the provider groups, 
respondents registered very large impacts on 
knowledge, with large impacts on their prac- 
tices for all but the WVDE, which missed 
that threshold by .1 to .2 points, placing it in 
the moderate impacts category. Respond- 
ents registered similar moderate change in 
their attitudes and beliefs, regardless of 
which provider group’s session they attend- 
ed. 



Figure 11. Mean Perceived Impact of 

Professional Development, Pre-/ 
Postsession, Total Sample 
Scale points ranged from 1 (not at all), 2 (to 
a small extent), 3 (to a moderate extent), 
and 4 (to a great extent). 


Figure 13 displays the range of effects 
for individual providers, which generally fol- 
low the pattern described above, with larger 
effects for knowledge, more moderate effects 
for practice, and the smallest effects for atti- 
tudes and beliefs. 
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Figure 12. Perceived Impact of Professional 

Development ( Pre-/Postsession ), Effect 
Size , by Individual Provider 

Dark brown indicates a very large effect, dark red a 
large effect, and medium red a moderate effect. 


4 Only the attitudes test for the WVDE Office of Child Nutrition registered a lack of statistical 
significance, with p = .211; however these participants entered the training with relatively high atti- 
tude scores, so the small amount of change may not be of any practical significance, either. 
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Figure 13. Perceived Impact of Professional Development ( Pre-/Postsession ), Effect Size , by Individual Provider 
Light blue indicates small effects; medium blue, moderate effects, dark blue, large effects, dark navy blue, 
very large effects. 


Implementation of the Master Plan for Statewide Professional Staff Development for 2012-2013 / 23 



Discussion 


In one last, disaggregation, we compared impacts by duration of the training, that is, 
informational sessions lasting up to 4 hours, versus technical training lasting from 4 to 13 
hours, or sustained professional development lasting 14 or more hours (Figure 14). As one 
would expect, the size of the effect reported for all three measures (i.e., knowledge, practices, 
and attitudes/beliefs) was proportionate to the amount of contact time. Informational ses- 
sions registered the smallest effects followed by technical training, with largest effects re- 
ported for sustained professional development. 

Looking back at the past three years of this measure, Figure 15, shows only a slight 
gain for perceived impacts on knowledge, and slight decreases for impacts on practice and 
attitudes /beliefs . 
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Figure 14. Perceived impact of Professional 

Development (Pre-/Postsession), Effect 
Size , by Individual Provider 

Light orange indicates a small effect; medium orange, 
a moderate effect; orange, a large effect; and dark 
rust, a very large effect. 



Using Cohen's d, effect sizes have been calculated for 
the total group of respondents to the Participant 
Survey for the past three years. Results show little 
overall movement during this period. 


Discussion 

In this evaluation, as in previous years, we examined four main aspects of the im- 
plementation of the West Virginia Board of Education’s Master Plan for Statewide Profes- 
sional Development: (a) basic information reported by providers about the size and scope of 
the effort, including attendance, and adherence to the newly adopted standards for profes- 
sional development; and participant reports about the (b) quality of the sessions, (c) their 
alignment to Board goals for professional development, and (d) the impacts of the sessions 
on participants’ knowledge, practice, and attitudes and beliefs. Each of these four areas are 
discussed below, including trends noted over the three years that the WVDE Office of Re- 
search has conducted this evaluation. 

With regard to basic issues of implementation, by far the most notable trend was the 
decrease in participation in the PD Master Plan by the RESAs during this period. Before 
providing details about this decline it should be noted that this trend was reversed in the 
2013-2014 PD Master Plan (an evaluation of which is currently underway), making 2012- 
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2013 a low point. This shift will be covered in subsequent evaluation reports; however for the 
3-year time period from 2010-2011 through 2012-2013, 

• the Center for Professional Development (CPD) increased its slate of sessions more 
than fivefold; 

• institutions of higher education (IHEs) with teacher preparation programs held 
steady at a very low level of participation with only one (Marshall University) of 12 
participating; 

• the WVDE more than doubled its participation; and 

• the RESAs reduced their collective contribution to the PD Master Plan by about two 
thirds. 

As for attendance in professional development sessions offered by the four provider groups 
required to participate, the WVDE was responsible for more than three quarters of all partic- 
ipants in PD Master Plan sessions in 2012-2013. 

RESA directors indicated on multiple occasions following the publication of the 
2010-2011 evaluation report that one insurmountable impediment to their participation was 
the schedule they were required to follow in submitting their lists of sessions for inclusion in 
the plan. They argued that because they must provide PD in response to the strategic plans 
of the districts they serve (submitted in early fall), they could not predict at the time the PD 
Master Plan was being developed, what PD they would need to offer. For this reason, the 
Board allowed all providers to update their plans in late fall, beginning in 2012. Marshall 
University and seven WVDE offices took this opportunity to add sessions to their plans; 
none of the RESAs did. 

In 2013-2014, however, RESAs seem to have changed their approach to the PD Mas- 
ter Plan. In that plan the RESAs vary in their number of offerings from a low of two sessions 
(RESA 8) to a high of 49 (RESA 1). The higher numbers of offerings by most RESAs more 
closely reflect their reports for professional development in their annual reports. This is a 
situation that will need continued monitoring, as RESAs take a larger role in providing pro- 
fessional development, and as the State Board works to develop a more coherent statewide 
system for professional learning. 

Other notable implementation trends include the fact that nearly 7,400 educators in 
the state participated in sessions of 30 hours or more duration, which is the minimum that 
recent reviews of the research identify as producing changes in teacher practice and/or stu- 
dent performance. Of the sessions offered during the reporting period, about a third were 
brief, informational sessions, another third were half-day to slightly less that two-day tech- 
nical training sessions and the remaining third were sessions two or more days duration. 

This was the first year that providers were asked to report how aligned their offerings 
were with the new Board standards for professional development, which are an adaptation 
of the Learning Forward standards. Overall, there was less than a 60% level of compliance 
with the standards. The Center for Professional Development reported 100% compliance for 
all standards for all sessions, while Marshall University reported a rate of compliance at 
about 78%, followed by the RESAs at 67% and WVDE at 48%. By their own self-reports WV 
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providers, overall, are strongest with regard to the following Board professional learning 
standards, with which they reported about two-thirds of their sessions aligned: 

l. Occurs within learning communities committed to continuous improvement, collec- 
tive responsibility, and goal alignment. 

5. Integrates theories, research, and models of human learning into learning designs to 
achieve its intended outcomes. 

7. Aligns its outcomes with educator performance and student curriculum standards. 

Weakest alignment (less than half of reported sessions) with the Board professional learning 
standards was for the following: 

4. Uses a variety of sources and types of student, educator, and system data to plan, as- 
sess, and evaluate professional learning. 

6. Applies research on change and sustains support for implementation of professional 
learning for long-term change. 

There seemed to be some confirmation in the participant survey responses for the lack of 
alignment with Standard 6; only two thirds of participants agreed or strongly agreed that the 
professional development session they attended had included "adequate embedded follow- 
up and continuous feedback.” In other words, ongoing follow-up to help them succeed in 
their implementation was lacking in a third of participants' experiences. 

The standards are new for state providers, and although they were included in infor- 
mation that went out to providers during the time the PD Master Plan was developed, it is 
unclear how aware of them most providers are. Data about the Board standards in this re- 
port should be considered baseline, and we will follow trends regarding providers’ alignment 
with them in upcoming evaluation studies. Further, relying primarily on provider self- 
reports to measure alignment with Board standards for professional development is a limita- 
tion that should be noted. 

Turning now to perceived quality, alignment with Board goals, and impacts, we note 
the following trends: 

• This analysis showed no overall gain in quality since 2011-2012, with a score of 3.9 
on a 5-point scale both years; there was only a slight gain compared with 2010-2011, 
which saw a score of 3.8. 

• With regard to participants’ recognition that the professional development was 
helpful in meeting Board goals for professional development, the overall agreement 
rate of 76.1%, exceeded the previous two years (i.e., 2010-2011, 67.8% and 2011- 
2012, 51.2%) There may be three factors at work in the relative high rate experi- 
enced this year: (a) providers were guided to a greater degree than previous years by 
the Board goals as they planned their sessions; (b) the goals were written more 
broadly, so it was easier for participants to see the connections; and/or (c) providers 
were required to select only one goal as aligned to the offerings in the PD Master 
Plan and were, therefore, less likely to select multiple, less closely tied goals for indi- 
vidual offerings. 

• Although effect sizes ranged from moderate to very large, there was only a slight 
gain for perceived impacts on knowledge, and slight decreases for impacts on prac- 
tice and attitudes/beliefs. 
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Taken together, these results show general satisfaction with the professional devel- 
opment participants experienced, but do not show much movement in improving the quality 
and impact. Further, the notable improvement in alignment with Board goals may have 
more to do with the goals themselves than with providers’ efforts to align their offerings. 

Limitations of the Study 

The participant survey conducted in November-December 2012 and April-May 2013 
asked respondents to recall PD sessions they had participated in at some point in the past. In 
some cases, the sessions had taken place up to five months prior to the survey. For this rea- 
son, there is a possibility of temporal bias in survey participants’ responses. 

Furthermore, the use of a retrospective pretest/posttest methodology to assess 
changes in knowledge, behavior and skills, and attitudes and beliefs poses some concerns. 
We used this methodology primarily because some researchers have argued that a phenom- 
enon called response shift bias can occur when conducting traditional pretest/posttest de- 
signs. Response-shift bias “occurs when a participant uses a different internal understanding 
of the construct being measured to complete the pretest and posttest” (Moore & Tananis, 
2009, p. 190). Consider this in context of professional development. Some respondents 
begin their involvement in professional development with a misconception that they are al- 
ready well-versed in the content to be covered. When given a pretest, they rate their own 
knowledge, behavior and skills, and attitudes and beliefs very positively. However, over the 
course of the professional development, as they develop a deeper understanding of the con- 
tent being covered, they realize they did not know as much as they originally thought. As 
such, when presented with the posttest, their frame of reference has shifted and they could 
potentially rate their knowledge, behavior and skills, and attitudes and beliefs lower than 
they did on the pretest. This can lead to problems in analyzing the impact of the professional 
development. For this reason, some researchers advocate for using retrospective pre- 
test/posttest designs as we did in this study. 

Despite this strength of the retrospective pretest/posttest design, a recent research 
study conducted by Nimon, Zigarmi, and Allen (2011) found that using traditional pre- 
test/posttest designs leads to less biased estimates of program effectiveness. The authors 
present a compelling case that presenting both pre- and posttest items simultaneously on a 
single survey is among the most biased design options available to researchers and can sig- 
nificantly inflate effect size estimates. The authors recommend traditional pretest/posttest 
designs when possible and advocate for the implementation of a separate retrospective pre- 
test to allow researchers to determine the presence of any response-shift bias. This design 
option, despite its strength, was not feasible in this study due to a mismatch between the 
scale of professional development offerings in the state and available evaluation staffing re- 
sources. Therefore, we recommend cautious interpretation of our own estimates of effect 
size, as they may be somewhat inflated. 
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Recommendations 

As this report is written, the West Virginia Board of Education has engaged the Na- 
tional Commission on Teaching and America's Future to lead an effort to overhaul the state's 
approach to professional development. Recommendations in previous evaluations of the 
Board's Master Plan for Statewide Professional Development will likely be addressed in the 
course of this overhaul. In the meantime we offer the following recommendations: 

• Find ways to increase the participation of institutions of higher education with teach- 
er preparation programs from the current one IHE (Marshall University) to the full 
12 IHEs that should be a part of it. 

• Consider developing goals for professional development with a longer view, commit 
to those goals for a sustained period of time and publicize them broadly, so that those 
planning for and providing professional development at all levels will be fully aware 
of them and willing to align their efforts to form a more coherent statewide approach. 

• Provide information about the Board standards for professional learning to all pro- 
fessional development providers working in the state, and develop training and in- 
centives that will compel providers to craft their offerings to meet those standards. 
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Table 4. Provision of Professional Development Included in the PD Master Plan and Attendance, by 


Provider 


Provider 

PD planned 

PD 

provided 

PD not 
provided* 

Percent of 
planned 
PD 

provided 

Individual 

sessions Attendance 
reported all sessions 

All providers 

383 

311 

68 

81.2 

1,018 

32,582 

Center for Professional Development 
Marshall Univ. Clinical Experiences 

57 

52 

5 

91.2 

110 

2,818 

9 

5 

4 

55.6 

7 

120 

and PD Schools 

Marshall Univ. June Harless Center 

17 

17 


100.0 

28 

719 

RESA 1 

3 

3 


100.0 

41 

912 

RESA2 

3 

3 


100.0 

9 

259 

RESA 3 

3 

3 


100.0 

13 

566 

RESA 4 

3 

3 


100.0 

19 

460 

RESA 5 

3 

3 


100.0 

7 

200 

RESA 6 

3 

3 


100.0 

24 

708 

RESA 7 

3 

3 


100.0 

24 

798 

RESA 8 

3 

3 


100.0 

9 

187 

WVDE Office of Assessment and 

29 

16 

9 

55.2 

59 

1,130 

Accountability 

WVDE Office of Career and Technical 

14 

9 

5 

64.3 

31 

1,268 

Accountability and Support 

WVDE Office of Career and Technical 

3 

3 


100.0 

3 

115 

Innovations 

WVDE Office of Career and Technical 

58 

43 

15 

74.1 

74 

1,587 

Instruction 

WVDE Office of Child Nutrition 

1 

1 


100.0 

1 

70 

WVDE Office of Early Learning 

10 

10 


100.0 

49 

2,624 

WVDE Office of Federal Programs 

9 

1 

8 

11.1 

6 

62 

WVDE Office of Healthy Schools 

3 

3 


100.0 

5 

369 

WVDE Office of Instruction 

14 

12 

2 

85.7 

38 

1,577 

WVDE Office of Instructional 

85 

74 

11 

87.1 

270 

6,802 

Technology 

WVDE Office of Optional Educational 

9 

5 

4 

55.6 

8 

596 

Pathways 

WVDE Office of Professional 

5 

5 


100.0 

27 

2,166 

Preparation 

WVDE Office of School Improvement 

6 

5 

1 

83.3 

24 

1,244 

WVDE Office of Special Programs 

30 

26 

4 

86.7 

132 

5,225 

*PD was considered "not provided" if there were no reports submitted under an individual session title. 
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Table 5. Timespan and Duration of Professional Development Sessions Offered by PD Master Plan Providers 

Informa- Recom- 

Avg. Avg. tional Sustained mended 

time- dura- Total (up to 4 Technical (14-29 (30 hours or 


tion sessions hours) (5-13 hours) hours) more) 


Organization: 

(days) 

(hours) 

N 

N 

% 

N 

% 

N 

% 

N 

% 

All providers 

21.3 

16.2 

1,018 

335 

32.9 

353 

34.7 

92 

9.0 

238 

23.4 

Center for Professional 

Development 

23.9 

20.2 

110 

0 

0.0 

55 

50.0 

2 

1.8 

53 

48.2 

Marshall Univ. Clinical Experiences 
and PD Schools 

146.3 

11.1 

7 

1 

14.3 

4 

57.1 

2 

28.6 

0 

0.0 

Marshall Univ. June Harless Center 

103.1 

36.0 

28 

7 

25.0 

4 

14.3 

6 

21.4 

11 

39.3 

RESA1 

14.7 

9.3 

41 

23 

56.1 

10 

24.4 

3 

7.3 

5 

12.2 

RESA2 

1.3 

8.2 

9 

3 

33.3 

3 

33.3 

3 

33.3 

0 

0.0 

RESA3 

1.2 

4.8 

13 

9 

69.2 

4 

30.8 

0 

0.0 

0 

0.0 

RESA4 

127.5 

23.6 

19 

0 

0.0 

6 

31.6 

6 

31.6 

7 

36.8 

RESA5 

45.1 

29.7 

7 

0 

0.0 

3 

42.9 

1 

14.3 

3 

42.9 

RESA6 

1.8 

6.5 

24 

15 

62.5 

6 

25.0 

2 

8.3 

1 

4.2 

RESA7 

10.5 

9.2 

24 

6 

25.0 

12 

50.0 

5 

20.8 

1 

4.2 

RESA8 

14.6 

10.9 

9 

0 

0.0 

8 

88.9 

0 

0.0 

1 

11.1 

WVDE Office of Assessment and 
Accountability 

1.0 

4.9 

59 

32 

54.2 

25 

42.4 

2 

3.4 

0 

0.0 

WVDE Office of Career and 

Technical Accountability 

1.4 

6.9 

31 

14 

45.2 

11 

35.5 

6 

19.4 

0 

0.0 


WVDE Office of Career and 6.0 38.3 3 0 0.0 1 33.3 1 33.3 1 33.3 


Technical Innovations 

WVDE Office of Career and 6.0 6.1 74 53 71.6 9 12.2 10 13.5 2 2.7 


Technical Instruction 


WVDE Office of Child Nutrition 

WVDE Office of Early Learning 

1.0 

11.9 

6.0 

5.3 

1 

49 

0 

18 

0.0 

36.7 

1 

28 

100.0 

57.1 

0 

3 

0.0 

6.1 

0 

0 

0.0 

0.0 

WVDE Office of Federal Programs 

1.3 

4.8 

6 

5 

83.3 

0 

0.0 

1 

16.7 

0 

0.0 

WVDE Office of Healthy Schools 

2.2 

13.8 

5 

1 

20.0 

2 

40.0 

2 

40.0 

0 

0.0 

WVDE Office of Instruction 

2.9 

14.9 

38 

0 

0.0 

26 

68.4 

6 

15.8 

6 

15.8 

WVDE Office of Instructional 

26.2 

26.8 

270 

95 

35.2 

32 

11.9 

6 

2.2 

137 

50.7 

Technology 

WVDE Office of Optional 

1.8 

8.8 

8 

3 

37.5 

3 

37.5 

2 

25.0 

0 

0.0 

Educational Pathways 

WVDE Office of Professional 

3.9 

4.8 

27 

15 

55.6 

11 

40.7 

1 

3.7 

0 

0.0 

Preparation 

WVDE Office of School 

28.2 

17.0 

24 

0 

0.0 

19 

79.2 

3 

12.5 

2 

8.3 

Improvement 

WVDE Office of Special Programs 

17.0 

12.8 

132 

35 

26.5 

70 

53.0 

19 

14.4 

8 

6.1 
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Table 6. Number and Percentage of Participants in Professional Development by Duration by Provider 



All 

Informational 
(up to 4 hours) 

Technical 
(5-13 hours) 

Sustained 
(14-29 hours) 

Recommended 
(30 hours plus) 

Organization 

sessions 

N 

% 

N 

% 

N 

% 

N 

% 

All providers 

32,582 

8,980 

27.6 

11,349 

34.8 

4,864 

14.9 

7,389 

22.7 

Center for Professional 

2,818 

0 

0.0 

1,477 

52.4 

18 

0.6 

1,323 

46.9 

Development 

Marshall University Clinical 

120 

7 

5.8 

79 

65.8 

34 

28.3 


0.0 

Experiences and Professional 

Development Schools 

Marshall University June Harless 

719 

116 

16.1 

56 

7.8 

194 

27.0 

353 

49.1 

Center 

RESA 1 

912 

397 

43.5 

250 

27.4 

73 

8.0 

192 

21.1 

RESA2 

259 

58 

22.4 

78 

30.1 

123 

47.5 


0.0 

RESA 3 

566 

179 

31.6 

387 

68.4 

0 

0.0 


0.0 

RESA 4 

460 

0 

0.0 

255 

55.4 

84 

18.3 

121 

26.3 

RESA 5 

200 

0 

0.0 

125 

62.5 

20 

10.0 

55 

27.5 

RESA 6 

708 

314 

44.4 

255 

36.0 

116 

16.4 

23 

3.2 

RESA 7 

798 

63 

7.9 

504 

63.2 

226 

28.3 

5 

0.6 

RESA 8 

187 

0 

0.0 

181 

96.8 

0 

0.0 

6 

3.2 

WVDE Office of Assessment and 

1,130 

562 

49.7 

480 

42.5 

88 

7.8 


0.0 

Accountability 

WVDE Office of Career and 

1,268 

522 

41.2 

414 

32.6 

332 

26.2 


0.0 

Technical Accountability and 

Support 

WVDE Office of Career and 

115 

0 

0.0 

70 

60.9 

39 

33.9 

6 

5.2 

Technical Innovations 

WVDE Office of Career and 

1,410 

1,170 

83.0 

0 

0.0 

199 

14.1 

41 

2.9 

Technical Instruction 

WVDE Office of Child Nutrition 

247 

0 

0.0 

247 

100.0 

0 

0.0 


0.0 

WVDE Office of Early Learning 

2,624 

834 

31.8 

1,519 

57.9 

271 

10.3 


0.0 

WVDE Office of Federal Programs 

62 

33 

53.2 

0 

0.0 

29 

46.8 


0.0 

WVDE Office of Healthy Schools 

369 

150 

40.7 

49 

13.3 

170 

46.1 


0.0 

WVDE Office of Instruction 

1,577 

0 

0.0 

421 

26.7 

172 

10.9 

984 

62.4 

WVDE Office of Instructional 

6,802 

2,589 

38.1 

803 

11.8 

342 

5.0 

3,068 

45.1 

Technology 

WVDE Office of Optional 

596 

17 

2.9 

129 

21.6 

450 

75.5 


0.0 

Educational Pathways 

WVDE Office of Professional 

2,166 

833 

38.5 

1,315 

60.7 

18 

0.8 


0.0 

Preparation 

WVDE Office of School 

1,244 

0 

0.0 

350 

28.1 

546 

43.9 

348 

28.0 

Improvement 

WVDE Office of Special Programs 

5,225 

1,136 

21.7 

1,905 

36.5 

1,320 

25.3 

864 

16.5 
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Table 7. Attendance by Location 


Location 

Attendance 

Location 

Attendance 

All locations 



32,582 

Multiple locations 

2,106 

McDowell 

47 

Multiple and online 

359 

Mercer 

206 

PD was online 

4,343 

Mineral 

222 

Barbour 

53 

Mingo 

298 

Berkeley 

775 

Monongalia 

2,675 

Boone 

131 

Monroe 

93 

Braxton 

410 

Morgan 

4 

Brooke 

89 

Nicholas 

637 

Cabell 

1,451 

Ohio 

762 

Calhoun 

0 

Pendleton 

0 

Clay 

28 

Pleasants 

57 

Doddridge 

34 

Pocahontas 

219 

Fayette 

181 

Preston 

97 

Gilmer 

220 

Putnam 

511 

Grant 

72 

Raleigh 

1,756 

Greenbrier 

80 

Randolph 

125 

Hampshire 

443 

Ritchie 

23 

Hancock 

113 

Roane 

57 

Hardy 

16 

Summers 

176 

Harrison 

1,758 

Taylor 

159 

Jackson 

134 

Tucker 

136 

Jefferson 

336 

Tyler 

0 

Kanawha 

7,882 

Upshur 

17 

Lewis 

493 

Wayne 

306 

Lincoln 

290 

Webster 

16 

Logan 

281 

Wetzel 

65 

Marion 

503 

Wirt 

36 

Marshall 

181 

Wood 

929 

Mason 

106 

Wyoming 

85 
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Table 8. Alignment with Board Standards for Professional Development by Standard by Provider 
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Appendix A. Tables 


Table 9. Roles of respondents 


Role 

Number 

Percent 

All role-group respondents 

5,958 

100.0 

Classroom teacher 

2,865 

48.1 

District central office staff 

502 

8.4 

Instructional support teacher (non-special 
education) 

216 

3.6 

Other 

984 

16.5 

Paraprofessional/aide 

29 

0.5 

Principal/assistant principal 

646 

10.8 

RESA staff 

67 

1.1 

Special education teacher 

649 

10.9 
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Appendix A. Tables 


Table 10. 

Number of Participant Survey Respondents by County or Other Employer 


Employer 

Number Employer 

Number 


Higher Education 

48 

Mason County Schools 

44 

Other 

130 

McDowell County Schools 

53 

Out of state 

22 

Mercer County Schools 

105 

Regional Education Service Agency 

41 

Mineral County Schools 

41 

WV Department of Education 

54 

Mingo County Schools 

86 

Barbour County Schools 

41 

Monongalia County Schools 

136 

Berkeley County Schools 

182 

Monroe County Schools 

44 

Boone County Schools 

74 

Morgan County Schools 

33 

Braxton County Schools 

31 

Nicholas County Schools 

56 

Brooke County Schools 

51 

Ohio County Schools 

55 

Cabell County Schools 

209 

Pendleton County Schools 

26 

Calhoun County Schools 

18 

Pleasants County Schools 

33 

Clay County Schools 

32 

Pocahontas County Schools 

21 

Doddridge County Schools 

39 

Preston County Schools 

56 

Fayette County Schools 

114 

Putnam County Schools 

109 

Gilmer County Schools 

36 

Raleigh County Schools 

114 

Grant County Schools 

33 

Randolph County Schools 

54 

Greenbrier County Schools 

84 

Ritchie County Schools 

22 

Hampshire County Schools 

51 

Roane County Schools 

35 

Hancock County Schools 

54 

Summers County Schools 

29 

Hardy County Schools 

24 

Taylor County Schools 

41 

Harrison County Schools 

137 

Tucker County Schools 

18 

Institutional Education Program 

23 

Tyler County Schools 

22 

Jackson County Schools 

47 

Upshur County Schools 

49 

Jefferson County Schools 

149 

Wayne County Schools 

71 

Kanawha County Schools 

322 

Webster County Schools 

26 

Lewis County Schools 

41 

Wetzel County Schools 

48 

Lincoln County Schools 

91 

Wirt County Schools 

26 

Logan County Schools 

65 

Wood County Schools 

140 

Marion County Schools 

86 

WV Schools for the Deaf and the Blind 

17 

Marshall County Schools 

60 

Wyoming County Schools 
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Table 13. Percent of Respondents Who Agree That the Session Was Helpful in Meeting the Designated Board Goal for Professional Development 

Goal 1 Agree Goal 2A Agree Goal 2B Agree Goal 3 Agree 

Non- Non- Non- Non- 

Total Agree agree Percent Total Agree agree Percent Total Agree agree Percent Total Agree agree Percent 
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Goal 1 Agree Goal 2A Agree Goal 2B Agree Goal 3 Agree 
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Total Agree agree Percent Total Agree agree Percent Total Agree agree Percent Total Agree agree Percent 
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Table 14. Perceived Impact of Professional Development (Pre-/Postsession) Overall and by Provider Group 

Treatment Group Comparison Group Pre/Posttest Effect size 

Provider Dimension Mean SD n Mean SD n Correlation Sig (Cohen's d) 

All providers Knowledge 3.58 .575 5446 2.83 .774 5446 .427 .000 1.1818 


T — 1 

O 


T — 1 

LD 

rsi 

00 

03 

03 


oo 

03 

LO 

LO 



T — 1 

LD 

rsj 

LD 

00 

00 

o 

LO 




o 

LD 


rsj 


T— 1 


^|- 

O 

LD 

00 

00 


rsi 

rsi 


CM 


T— 1 


oo 

o 

LD 

LD 




LO 

oo 


LO 

LD 

LD 

rsi 

03 


T— 1 


LO 

LD 

oo 

oo 


LO 

LD 

LD 

oo 

oo 

t-H 

LO 

LO 

00 




LO 


rsi 

00 

LO 



LO 

rsi 


LO 

T—\ 


LO 


rsj 

00 

LO 


T— 1 

LO 



LO 



o 

o 


T — 1 

o 

o 

T — 1 

o 

o 

T — 1 

o 

o 

T — 1 

o 

o 


T — 1 

o 

o 

T — 1 

T— 1 

o 

T — 1 

o 

o 



o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 



o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 



o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 



ro 

03 


LO 

rsi 

00 



T— 1 

oo 

T— 1 

o 

o 

LO 

03 


LO 

rsi 

00 

T— 1 

03 

03 

rsj 

00 

03 




LD 


LO 

o 

rsi 

o 

oo 

LD 

LO 


00 

oo 


LD 


LO 

o 

rsj 

00 

oo 

o 

oo 

ro 




LO 

LD 


ro 

LO 

LO 


LO 

LO 


LO 

LO 


LD 

LO 


oo 

LO 

LO 

rsi 


LD 


LO 

LO 



LO 

rsi 


00 

03 

00 

oo 


O 

o 

G3 

rsi 

LO 

o 

rsi 


00 

03 

00 

00 

00 

00 

LO 

03 

CM 



00 

T— 1 


LD 

LO 

oo 

03 

00 

03 


T— 1 

rsi 


rsi 

LD 


LD 

LO 

oo 





ro 




T— 1 

rsi 


LD 

LD 

LD 

T— 1 

r—\ 

r—\ 

LD 

LD 

LD 

03 




LD 

LD 

LD 




T— 1 

rH 

tH 



LO 

LO 











oo 

oo 

oo 













00 

03 


ro 


03 

00 

rsi 

rsi 

T— 1 

LD 

03 

T— 1 

T— 1 

T— 1 


OO 


03 


i^ 

LO 



o 



o 

rsi 



LO 

oo 

LO 

rsj 

rsj 

o 

LO 

rsi 

00 

rsi 




LO 

oo 

T— 1 


oo 

O 

LD 

LO 



00 





LD 

00 

00 






00 





LD 

LD 

LD 

LD 

03 

00 



oo 






















































O 

r\j 

LD 


no 


T — 1 

LD 


00 

00 

00 

03 

T — 1 

T — 1 

00 


oo 


T — 1 

LD 

00 

LO 

LD 

03 

LD 



03 

rsj 


O 

o 


LD 

00 

rsj 


00 

r — i 

00 

03 

rsj 


o 

o 

^1; 

G 3 

o 

00 

q 

q 

rsj 

Qj 

rsj 

rsi 

ro 


co 

ro 

oo 

rsi 

rsi 

oo 

rsi 

rsi 

oo 

rsi 

rsi 

oo 


ro 

oo 

oo 

rsi 

oo 

oo 

rsi 

rsi 

oo 

03 

^H 


























C3 

O 
















C/3 










CL 

<N 



> 













DC 










4—i 

L. 



cc 













ID 










>< 

o 

LO 

rsi 

O 

(Z> 

00 

03 

00 

oo 


O 

O 

G3 

rsi 

LO 

O 

rsi 

9 

> 

00 

03 

00 

00 

00 

00 

LO 

03 

CM 

03 

c 

4—i 

d 

00 

rH 

LD 

LD 

LO 

oo 

03 

00 

03 


T— 1 

rsi 


rsi 

LD 

o 

LD 

LO 

oo 





ro 


c: 

03 

i 

C\J 

1— 

LD 

LD 

LD 

T— 1 

i 

T— 1 

LD 

LD 

LD 

03 



LD 

LD 

LD 




T— 1 

rH 

t-H 

o 

E 

LO 

LO 

< 










oo 

oo 

oo 

oc 










L0 



LJ 













CL 










03 

CL 



cc 

LD 

9 













1 

< 

z> 










.c 

4C 

c 

~Qj 

03 


LD 

> 

03 

00 

rsi 

T— 1 

03 

oo 

00 

00 

O 

LO 


o 

i i 

03 

00 

rsi 

LD 

00 

CM 

o 

O 

LO 

o 

Q 


03 


03 


03 

LO 

LO 




T— 1 

00 

00 

T— 1 

> 

03 


03 

rsi 

LD 

rH 

LD 

O 

LO 

o 

t 

LD 

LD 

CJ 

cd 

CL 


LO 


LO 

LD 

LO 

LO 

LD 

LD 

LO 

LD 

LD 

Cl 

2 


LO 


LO 


LO 

LO 


LO 

_Qj 






















































C3 



























C 

o 

T — 1 

00 


rsi 

LD 

^l - 

o 

00 


rsi 

LD 

O 

r^ 

03 

rsj 


rsi 

LD 


LO 

03 

03 

03 

T — 1 

r^ 


LH 

UD 



LD 

1^ 

LD 


LD 

LH 


LD 

LO 


LD 



q 

1^ 

q 

q 

q 

q 


q 



ro 

00 


ro 

00 

oo 

OO 

oo 

oo 

CO 

oo 

00 

oo 

CO 

00 


ro 

oo 

oo 

co 

oo 

oo 

co 

ro 

oo 





























8 



























CL 



























0) 





























to 




to 



to 



to 



to 




to 



to 



to 


i 


M— 




M— 



M— 



M— 



M— 




M— 



M— 



M— 


0) 

4-i 


03 




cu 



03 



03 



03 




03 



03 



03 



"aj 




"aj 



"aj 



"aj 



"aj 




"aj 



"aj 



"aj 


2 


_Q 


a> 


_Q 

O) 


_Q 

03 


_Q 

03 


_Q 


03 


_Q 

03 


_Q 

03 


_Q 


to 

to 

03 

u 

to 

03 

"a 


"O 

cu 

to 

03 

u 

to 

a) 

"a 

00 

"D 

03 

to 

a > 
u 

to 

03 

"D 

00 

"D 

03 

to 

03 

u 

to 

03 

"a 

00 

"D 

03 

to 

03 

U 

to 

03 

"a 


CUD 

"O 

03 

to 

03 

u 

to 

03 

"C3 

00 

"D 

03 

to 

03 

u 

to 

03 

"D 

00 

"D 

03 

to 

03 

CJ 

to 

03 

"a 


L. 

>2. 

r— 

'+- > 
u 
ro 

d 


<S 

o 

4-> 

o 

03 

Z3 

4-» 

£ 

o 

'+-* 

o 

03 

13 

+J 

4-» 

£ 

o 

4—* 

o 

03 

13 

+J 

4— ’ 

<s 

o 

*4—* 

U 

03 

13 

+J 

4— ’ 


o 

4—* 

O 

03 

u 

4-J 

4— ’ 

o 

4—* 

O 

03 

D 

4-J 

4— ’ 

o 

*4—* 

U 

CO 

D 

4-J 

4— ’ 


5l 

i— 

-t-j 


c 

t — 

4—* 

c 

i— 

4— * 

c 

i— 

4—* 

c 


4—* 


c 

i— 

4—* 

c 

t_ 

4—* 

c 


4—* 



CL 

< 



Cl 

< 


CL 

< 


CL 

< 


Q_ 

< 



CL 

< 


CL 

< 


CL 

< 


L. 

03 



























4-> 























to 

to 

_03 




to 




















"D 

c 






C3 

§ 




















03 

75 



CD 

X 




0) 

2 




















CJ 



03 
























'c 



c 




o 




(TS 

c 













CO 

c 



U 



D 




c: 

o 

■iC 




o 













o 



>* 



>- 







to 

+-> 












to 

4—* 


4-J 







C3 




to 

c 












to 

c 


'to 



'to 




+3 




03 

4— 

O 

a> 

E 












03 

M— 

O 

03 

E 


03 

> 

to 

o 


s— 

03 

> 




C 

0) 




L_ 

Q_ 

Q. 

O 












t_ 

CL 

Q. 

o 


”c 

Z) 

O 

_c 


'c 

3 

03 



E 

_Qj 




O 

CU 












o 

03 




C/3 




4—* 



Cl 




M— 

> 












M— 

> 


— 


— 

c 



E 




S— 

a> 













03 


ro 

Cl 


ro 

03 






03 

+-> 

Q 





to 

< 



LU 

O 




03 

4— 1 

Q 


_c 

to 

CL 


_c 

to 

u 








c 



D 



oo 



> 




c 



c5 



co 








a> 

u 



X 



ID 

cd 



5 




03 

u 










o 


Table 14. Perceived Impact of Professional Development (Pre-/Postsession) Overall and by Provider Group 

Treatment Group Comparison Group Pre/Posttest Effect size 

Provider Dimension Mean SD n Mean SD n Correlation Sig (Cohen's d) 

RESA1 Knowledge 3.43 .582 90 2.66 .752 90 .474 .000 1.1872 
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Table 14. Perceived Impact of Professional Development (Pre-/Postsession) Overall and by Provider Group 

Treatment Group Comparison Group Pre/Posttest Effect size 

Provider Dimension Mean SD n Mean SD n Correlation Sig (Cohen's d) 
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Table 14. Perceived Impact of Professional Development (Pre-/Postsession) Overall and by Provider Group 

Treatment Group Comparison Group Pre/Posttest Effect size 

Provider Dimension Mean SD n Mean SD n Correlation Sig (Cohen's d) 
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